Equipment employing watermark-based authentication function

ABSTRACT

Image, video, or audio data is encoded with both a frail and a robust watermark. The two watermarks respond differently to different forms of processing (e.g., copying the object may render the frail watermark unreadable), permitting an original object to be distinguished from a processed object. Appropriate action can then taken in response thereto.

RELATED APPLICATION DATA

This application is a divisional of application Ser. No. 09/498,223 (nowU.S. Pat. No. 6,574,350), filed Feb. 3, 2000. The 09/498,223 applicationis a continuation-in-part of application Ser. No. 09/287,940 (now U.S.Pat. No. 6,580,819), filed Apr. 7, 1999, which claims priority toabandoned application Ser. No. 60/082,228, filed Apr. 16, 1998. The Ser.No. 09/498,223 application is also a continuation of application Ser.No. 09/433, 104 (now U.S. Pat. No. 6,636,615), filed Nov. 3, 1999, whichis a continuation-in-part of application Ser. No. 09/234,780, filed Jan.20, 1999 (now U.S. Pat. No. 6,332,031), which claims priority toabandoned application Ser. No. 60/071,983, filed Jan. 20, 1998. Theseapplications are incorporated herein by reference.

FIELD OF THE INVENTION

The present application relates to digital watermarking, andparticularly relates to digital watermarking techniques employing bothfrail and robust watermarks.

BACKGROUND AND SUMMARY OF THE INVENTION

For expository convenience, the following discussion focuses on anexemplary application of the disclosed technology—encoding the imagesprinted on banknotes with both frail and robust watermarks. As notedlater, however, the technology also finds application beyond imagewatermarking, including in video and audio watermarking.

The problem of casual counterfeiting of banknotes first arose twodecades ago, with the introduction of color photocopiers. A number oftechniques were proposed to address the problem.

U.S. Pat. No. 5,659,628 (assigned to Ricoh) is one of several patentsnoting that photocopiers can be equipped to recognize banknotes andprevent their photocopying. The Ricoh patent particularly proposed thatthe red seal printed on Japanese yen notes is a pattern well-suited formachine recognition. U.S. Pat. No. 5,845,008 (assigned to Omron), andU.S. Pat. Nos. 5,724,154 and 5,731,880 (both assigned to Canon) showother photocopiers that sense the presence of the seal emblem onbanknotes, and disable a photocopier in response.

Other technologies proposed that counterfeiting might be deterred byuniquely marking the printed output from each color photocopier, so thatcopies could be traced back to the originating machine. U.S. Pat. No.5,568,268, for example, discloses the addition ofessentially-imperceptible patterns of yellow dots to printed output; thepattern is unique to the machine. U.S. Pat. No. 5,557,742 discloses arelated arrangement in which the photocopier's serial number is printedon output documents, again in essentially-imperceptible form (smallyellow lettering). U.S. Pat. No. 5,661,574 shows an arrangement in whichbits comprising the photocopier's serial number are represented in thephotocopier's printed output by incrementing, or decrementing, pixelvalues (e.g. yellow pixels) at known locations by fixed amounts (e.g.+/−30), depending on whether the corresponding serial number bit is a“1” or a “0.”.

Recent advances in color printing technology have greatly increased thelevel of casual counterfeiting. High quality scanners are now readilyavailable to many computer users, with 300 dpi scanners available forunder $100, and 600 dpi scanners available for marginally more.Similarly, photographic quality color ink-jet printers are commonlyavailable from Hewlett-Packard Co., Epson, etc. for under $300.

These tools pose new threats. For example, a banknote can be doctored(e.g. by white-out, scissors, or less crude techniques) toremove/obliterate the visible patterns on which prior art banknotedetection techniques relied to prevent counterfeiting. Such a doctoreddocument can then be freely scanned or copied, even on photocopiersdesigned to prevent processing of banknote images. The removedpattern(s) can then be added back in, e.g. by use of digital imageediting tools, permitting free reproduction of the banknote.

In accordance with aspects of the present invention, these and othercurrent threats are addressed by digitally watermarking banknotes, andequipping devices to sense such watermarks and respond accordingly.(Watermarking is a quickly growing field of endeavor, with severaldifferent approaches. The present assignee's work is reflected in theearlier-cited related applications, as well as in U.S. Pat. Nos.5,841,978, 5,748,783, 5,710,834, 5,636,292, 5,721,788, and laid-open PCTapplication WO97/43736. Other work is illustrated by U.S. Pat. Nos.5,734,752, 5,646,997, 5,659,726, 5,664,018, 5,671,277, 5,687,191,5,687,236, 5,689,587, 5,568,570, 5,572,247, 5,574,962, 5,579,124,5,581,500, 5,613,004, 5,629,770, 5,461,426, 5,743,631, 5,488,664,5,530,759,5,539,735, 4,943,973, 5,337,361, 5,404,160, 5,404,377,5,315,098, 5,319,735, 5,337,362, 4,972,471, 5,161,210, 5,243,423,5,091,966, 5,113,437, 4,939,515, 5,374,976, 4,855,827, 4,876,617,4,939,515, 4,963,998, 4,969,041, and published foreign applications WO98/02864, EP 822,550, WO 97/39410, WO 96/36163, GB 2,196,167, EP777,197, EP 736,860, EP 705,025, EP 766,468, EP 782,322, WO 95/20291, WO96/26494, WO 96/36935, WO 96/42151, WO 97/22206, WO 97/26733. Some ofthe foregoing patents relate to visible watermarking techniques. Othervisible watermarking techniques (e.g. data glyphs) are described in U.S.Pat. Nos. 5,706,364, 5,689,620, 5,684,885, 5,680,223, 5,668,636,5,640,647, 5,594,809.

Most of the work in watermarking, however, is not in the patentliterature but rather in published research. In addition to thepatentees of the foregoing patents, some of the other workers in thisfield (whose watermark-related writings can by found by an author searchin the INSPEC database) include I. Pitas, Eckhard Koch, Jian Zhao,Norishige Morimoto, Laurence Boney, Kineo Matsui, A. Z. Tirkel, FredMintzer, B. Macq, Ahmed H. Tewfik, Frederic Jordan, Naohisa Komatsu, andLawrence O'Gorman.

The artisan is assumed to be familiar with the foregoing prior art.

In the present disclosure it should be understood that references towatermarking encompass not only the assignee's watermarking technology,but can likewise be practiced with any other watermarking technology,such as those indicated above.

The physical manifestation of watermarked information most commonlytakes the form of altered signal values, such as slightly changed pixelvalues, picture luminance, picture colors, DCT coefficients,instantaneous audio amplitudes, etc. However, a watermark can also bemanifested in other ways, such as changes in the surface microtopologyof a medium, localized chemical changes (e.g. in photographicemulsions), localized variations in optical density, localized changesin luminescence, etc. Watermarks can also be optically implemented inholograms and conventional paper watermarks.)

In accordance with an exemplary embodiment of the present invention, anobject-such as a banknote image-is encoded with two watermarks. One isrelatively robust, and withstands various types of corruption, and isdetectable in the object even after multiple generations of interveningdistortion. The other is relatively frail, so that it fails with thefirst distortion. If a version of the object is encountered having therobust watermark but not the frail watermark, the object can be inferredto have been processed, and thus not an original.

The foregoing and other features and advantages of the present inventionwill be more readily apparent from the following Detailed Description,which proceeds with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows part of an automatic teller machine employing principles ofthe present invention.

FIG. 2 shows part of a device (e.g. a photocopier, scanner, or printer)employing principles of the present invention.

FIG. 3 shows part of another device employing principles of the presentinvention.

DETAILED DESCRIPTION

Watermarks in banknotes and other security documents (passports, stockcertificates, checks, etc.—all collectively referred to as banknotesherein) offer great promise to reduce such counterfeiting, as discussedmore fully below. Additionally, watermarks provide a high-confidencetechnique for banknote authentication.

By way of example, consider an automatic teller machine that useswatermark data to provide high confidence authentication of banknotes,permitting it to accept—as well as dispense—cash. Referring to FIG. 1,such a machine (11) is provided with a known optical scanner (13) toproduce digital data (15) corresponding to the face(s) of the bill (16).This image set (14) is then analyzed (16) to extract embedded watermarkdata. In watermarking technologies that require knowledge of a codesignal (20) for decoding (e.g. noise modulation signal, crypto key,spreading signal, etc.), a bill may be watermarked in accordance withseveral such codes. Some of these codes are public—permitting theirreading by conventional machines. Others are private, and are reservedfor use by government agencies and the like. (C.f. public and privatecodes in the present assignee's issued patents.)

As noted, banknotes presently include certain visible structures, ormarkings (e.g., the seal emblem noted in the earlier-cited patents),which can be used as aids to note authentication (either by visualinspection or by machine detection). Desirably, a note is examined by anintegrated detection system (24), for both such visible structures (22),as well as the present watermark-embedded data, to determineauthenticity.

The visible structures can be sensed using known pattern recognitiontechniques. Examples of such techniques are disclosed in U.S. Pat. Nos.5,321,773, 5,390,259, 5,533,144, 5,539,841, 5,583,614, 5,633,952,4,723,149 and 5,424,807 and laid-open foreign application EP 766,449.The embedded watermark data can be recovered using the scanning/analysistechniques disclosed in the cited patents and publications.

To reduce counterfeiting, it is desirable that document-reproducingtechnologies recognize banknotes and refuse to reproduce same. Referringto FIG. 2, a photocopier (30), for example, can sense the presence ofeither a visible structure (32) or embedded banknote watermark data(34), and disable copying if either is present (36). Scanners andprinters can be equipped with a similar capability—analyzing the datascanned or to be printed for either of these banknote hallmarks. Ifeither is detected, the software (or hardware) disables furtheroperation.

The watermark detection criteria provides an important advantage nototherwise available. As noted, an original bill can be doctored (e.g. bywhite-out, scissors, or less crude techniques) to remove/obliterate thevisible structures. Such a document can then be freely copied on eithera visible structure-sensing photocopier or scanner/printer installation.The removed visible structure can then be added in via a secondprinting/photocopying operation. If the printer is not equipped withbanknote-disabling capabilities, image-editing tools can be used toinsert visible structures back into image data sets scanned from suchdoctored bills, and the complete bill freely printed. By additionallyincluding embedded watermark data in the banknote, and sensing same,such ruses will not succeed.

(A similar ruse is to scan a banknote image on a non-banknote-sensingscanner. The resulting image set can then be edited by conventionalimage editing tools to remove/obliterate the visible structures. Such adata set can then be printed—even on a printer/photocopier that examinessuch data for the presence of visible structures. Again, the missingvisible structures can be inserted by a subsequent printing/photocopyingoperation.)

Desirably, the visible structure detector and the watermark detector areintegrated together as a single hardware and/or software tool. Thisarrangement provides various economies, e.g., in interfacing with thescanner, manipulating pixel data sets for pattern recognition andwatermark extraction, electronically re-registering the image tofacilitate pattern recognition/watermark extraction, issuing controlsignals (e.g. disabling) signals to the photocopier/scanner, etc.

A related principle (FIG. 3) is to insert an imperceptible watermarkhaving a universal ID (UID) into all documents printed with a printer,scanned with a scanner, or reproduced by a photocopier. The UID isassociated with the particular printer/photocopier/scanner in a registrydatabase maintained by the products' manufacturers. The manufacturer canalso enter in this database the name of the distributor to whom theproduct was initially shipped. Still further, the owner's name andaddress can be added to the database when the machine is registered forwarranty service. While not preventing use of such machines incounterfeiting, the embedded UID facilitates identifying the machinethat generated a counterfeit banknote. (This is an application in whicha private watermark might best be used.)

While the foregoing applications disabled potential counterfeitingoperations upon the detection of either a visible structure orwatermarked data, in other applications, both criteria must be metbefore a banknote is recognized as genuine. Such applications typicallyinvolve the receipt or acceptance of banknotes, e.g. by ATMs asdiscussed above and illustrated in FIG. 1.

The foregoing principles (employing just watermark data, or inconjunction with visible indicia) can likewise be used to preventcounterfeiting of tags and labels (e.g. the fake labels and tagscommonly used in pirating Levis brand jeans, branded software, etc.)

The reader may first assume that banknote watermarking is effected byslight alterations to the ink color/density/distribution, etc. on thepaper. This is one approach. Another is to watermark the underlyingmedium (whether paper, polymer, etc.) with a watermark. This can be doneby changing the microtopology of the medium (a la mini-Braille) tomanifest the watermark data. Another option is to employ a laminate onor within the banknote, where the laminate has the watermarkingmanifested thereon/therein. The laminate can be textured (as above), orits optical transmissivity can vary in accordance with a noise-likepattern that is the watermark, or a chemical property can similarlyvary.

Another option is to print at least part of a watermark usingphotoluminescent ink. This allows, e.g., a merchant presented with abanknote, to quickly verify the presence of *some* watermark-likeindicia in/on the bill even without resort to a scanner and computeranalysis (e.g. by examining under a black light). Such photoluminescentink can also print human-readable indicia on the bill, such as thedenomination of a banknote. (Since ink-jet printers and other commonmass-printing technologies employ cyan/magenta/yellow/black to formcolors, they can produce only a limited spectrum of colors.Photoluminescent colors are outside their capabilities. Fluorescentcolors—such as the yellow, pink and green dyes used in highlightingmarkers—can similarly be used and have the advantage of being visiblewithout a black light.)

An improvement to existing encoding techniques is to add an iterativeassessment of the robustness of the mark, with a correspondingadjustment in a re-watermarking operation. Especially when encodingmultiple bit watermarks, the characteristics of the underlying contentmay result in some bits being more robustly (e.g. strongly) encoded thanothers. In an illustrative technique employing this improvement, awatermark is first embedded in an object. Next, a trial decodingoperation is performed. A confidence measure (e.g. signal-to-noiseratio) associated with each bit detected in the decoding operation isthen assessed. The bits that appear weakly encoded are identified, andcorresponding changes are made to the watermarking parameters to bringup the relative strengths of these bits. The object is then watermarkedanew, with the changed parameters. This process can be repeated, asneeded, until all of the bits comprising the encoded data areapproximately equally detectable from the encoded object, or meet somepredetermined signal-to-noise ratio threshold.

The foregoing applications, and others, can generally benefit bymultiple watermarks. For example, an object (physical or data) can bemarked once in the spatial domain, and a second time in the spatialfrequency domain. (It should be understood that any change in one domainhas repercussions in the other. Here we reference the domain in whichthe change is directly effected.)

Another option is to mark an object with watermarks of two differentlevels of robustness, or strength. The more robust watermark withstandsvarious types of corruption, and is detectable in the object even aftermultiple generations of intervening distortion. The less robustwatermark can be made frail enough to fail with the first distortion ofthe object. In a banknote, for example, the less robust watermark servesas an authentication mark. Any scanning and reprinting operation willcause it to become unreadable. Both the robust and the frail watermarksshould be present in an authentic banknote; only the former watermarkwill be present in a counterfeit.

Still another form of multiple-watermarking is with content that iscompressed. The content can be watermarked once (or more) in anuncompressed state. Then, after compression, a further watermark (orwatermarks) can be applied.

Still another advantage from multiple watermarks is protection againstsleuthing. If one of the watermarks is found and cracked, the otherwatermark(s) will still be present and serve to identify the object.

The foregoing discussion has addressed various technological fixes tomany different problems. Exemplary solutions have been detailed above.Others will be apparent to the artisan by applying common knowledge toextrapolate from the solutions provided above.

For example, the technology and solutions disclosed herein have made useof elements and techniques known from the cited references. Otherelements and techniques from the cited references can similarly becombined to yield further implementations within the scope of thepresent invention. Thus, for example, holograms with watermark data canbe employed in banknotes, single-bit watermarking can commonly besubstituted for multi-bit watermarking, technology described as usingimperceptible watermarks can alternatively be practiced using visiblewatermarks (glyphs, etc.), techniques described as applied to images canlikewise be applied to video and audio, local scaling of watermarkenergy can be provided to enhance watermark signal-to-noise ratiowithout increasing human perceptibility, various filtering operationscan be employed to serve the functions explained in the prior art,watermarks can include subliminal graticules to aid in imagere-registration, encoding may proceed at the granularity of a singlepixel (or DCT coefficient), or may similarly treat adjoining groups ofpixels (or DCT coefficients), the encoding can be optimized to withstandexpected forms of content corruption. Etc., etc., etc. Thus, theexemplary embodiments are only selected samples of the solutionsavailable by combining the teachings referenced above. The othersolutions necessarily are not exhaustively described herein, but arefairly within the understanding of an artisan given the foregoingdisclosure and familiarity with the cited art.

We claim:
 1. Electronic equipment including an authentication function,the equipment including a steganographic watermark detector that checkssampled data for the presence of first and second steganographicwatermarks, the first watermark being relatively frail and the secondwatermark being relatively robust, the equipment determining that thesampled data is authentic only if both watermarks are present.
 2. Theequipment of claim 1 wherein operation of the equipment is controlled ina first manner if both watermarks are present, and is controlled in asecond, different, manner if only one of the first and secondsteganographic watermarks is present.
 3. The equipment of claim 2,wherein the watermarking of at least the first watermark is effected byslight alterations to the sampled data's color.
 4. The equipment ofclaim 3, wherein the first watermark degrades with distortion, thedistortion comprising at least scanning and reprinting.
 5. Acash-accepting automatic teller machine that determines whether tenderedcash is authentic, according to claim
 1. 6. Electronic equipmentincluding an authentication function, the authentication functionincludes a steganographic watermark detector that checks sampled datafor the presence of first and second steganographic watermarks, whereinthe sampled data corresponds to a printed document, and wherein anoriginal printed document includes first and steganographic watermarkscooperating in an expected manner to evidence authenticity of theoriginal printed document, said electronic equipment determines that theprinted document is an original printed document only if both watermarksare present and if the watermarks evidence the expected cooperation. 7.The electronic equipment of claim 6, wherein the expected cooperationcomprises a relationship of relative robustness between the first andsecond watermarks.
 8. A method to differentiate copies of an originaldocument from the original document, wherein the original documentincludes a first digital watermark and a second digital watermark, saidmethod comprising: obtaining a first metric based on a comparison of thefirst and second digital watermarks in the original document; readingthe first and second watermarks from a copy of the original document andcomparing the watermarks to obtain a second metric; and usingdifferences between said first and second metrics to differentiate anoriginal document from a copy of the original document.